Image capturing apparatus for luminance correction, a control method therefor, and a recording medium

ABSTRACT

A low-frequency region of an output target image is specified. Luminance correction is performed with the target amount decided in advance for a pixel, among pixels included in the target image, which is included in the low-frequency region and has a luminance value equal to or smaller than a predetermined value, and the target image after the correction is output. Then, an effective correction amount is obtained based on the luminance distribution of the target image before the correction and the luminance distribution of the target image after the correction, which has been corrected with the target amount decided in advance. A luminance correction target amount for the next correction target image is decided based on the luminance distribution of the target image and the ratio of the effective correction amount to the target amount decided in advance.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image capturing apparatus, a controlmethod therefor, and a recording medium and, more particularly, to aluminance correction technique for successively captured images.

2. Description of the Related Art

An image capturing apparatus such as a digital camera or digital videocamera can correct a captured and obtained image to a desired state byapplying various image processes to the image. Correction processingapplied in the image capturing apparatus includes, for example,brightness correction, tone correction, and contrast correction. Mostimage capturing apparatuses determine the features of an obtained image,and apply those correction processes automatically or according to a setshooting mode or the like.

Correction processing for an image is applied to, for example, a backlitscene in which a light source exists in the background of an object. Ina backlit scene in which the sun is in the background of a person as amain object, for example, the surface of the main object which faces theimage capturing apparatus is shaded, thereby decreasing the luminance.In this case, by emitting flash light to shoot an image under exposureconditions where no blown-out highlight occurs in the background, it ispossible to obtain an image in a state in which the object and thebright background are preferable while ensuring the luminance of themain object. Using flash light as artificial light may make theluminance of the main object unnatural, or cause shadow-detail lossbecause the flash light does not reach the main object, depending on thedistance between the main object and the image capturing apparatus todecrease the luminance of the main object. In recent years, therefore, ashadow-detail loss region and a blown-out highlight region are reducedby applying, to an image shot in a backlit scene, image processing forperforming tone connection for a luminance signal. More specifically, asignal amplification ratio is changed according to the signal level(brightness) of each pixel or a tone level to be assigned is adjustedfor each signal level range to which each pixel belongs, therebyperforming correction.

If luminance correction is performed for all the pixels of an image asdescribed above, the following problems may occur depending on an objectin the image. FIG. 9A shows an image 901 captured in a backlit scene inwhich the sky and a tree are included in a shooting range. In the image901, a region corresponding to the sky has a luminance higher than thatof a region corresponding to the tree, and luminance gradation in whichthe luminance gradually decreases from the upper portion of the image tothe lower portion is seen. If luminance correction of assigning aluminance to each signal level range as described above is performed forall the pixels of the image, an image 902 shown in FIG. 9B is obtained.In the image 902, a boundary portion between the sky region with a highluminance and the tree region with a low luminance blurs. This isbecause the surrounding region of the tree is a so-called high-frequencycomponent region, where pixels representing branches and leaves of thetree with a low luminance are frequently mixed with pixels representingthe sky with a high luminance. In extracting a desired high-frequencycomponent region, whether each pixel existing near the boundary betweenbright and dark regions is a luminance correction target pixel ornon-target pixel is based, in part, on the region extractionperformance. If a pixel with a high luminance level to some extent whichshould not actually be a correction target is extracted as ahigh-frequency component region to be corrected, and is corrected to bebrighter, the boundary portion may blur.

A difference between correction results which have been obtained byperforming luminance correction for a low-frequency component region anda high-frequency component region, respectively, will be described withreference to FIGS. 10A to 10C. FIG. 10A shows an image 1000 having alow-frequency component as a whole, and an image 1010 having ahigh-frequency component as a whole. The images 1000 and 1010 are formedby high-luminance regions 1001 and 1011 with a median of 180 of theluminance distribution, and low-luminance regions 1002 and 1012 with amedian of 30 of the luminance distribution, respectively. In the image1000, there is one low-luminance region. To the contrary, in the image1010, there are a plurality of low-luminance regions, and there is ahigh-luminance region between the low-luminance regions. Thus, the image1010 has a high frequency component. In this case, the number of pixelsof the low-luminance region 1002 of the image 1000 is equal to the totalnumber of pixels of the low-luminance regions 1012 of the image 1010,and the luminance distributions of the images are equal to each other,as shown in FIG. 10B.

If luminance correction for increasing the luminance of thelow-luminance region is applied to the images 1000 and 1010, changes inluminance distributions of the images after the processing are differentfrom each other, as shown in FIG. 10C. More specifically, the median ofthe luminance distribution of the low-luminance region 1002 of the image1000 increases to 60, while that of the luminance distribution of thelow-luminance regions 1012 of the image 1010 increases to only 40. Thatis, the effect of luminance correction for the low-luminance regionchanges depending on whether the surrounding region of the target regionis a low-frequency component region or a high-frequency componentregion, as shown in FIG. 11. FIG. 11 shows that a correction amountafter actual correction for the high-frequency component region issmaller than that for the low-frequency component region for a givenluminance correction target amount; that is, only a small correctioneffect is produced for the former region.

On the other hand, in the image 902 of FIG. 9B, since the sky regionother the surrounding region of the tree is a low-frequency componentregion where the luminance gradually changes, a problem that, forexample, the edges blur does not arise. As shown in the image 902,however, it becomes impossible to reproduce the luminance gradation byassigning a luminance to each signal level range. Especially in a regionwith a high luminance, a luminance difference caused by tone assignmentreadily stands out, thereby giving the user the impression thatdegradation in tone has occurred in the corrected image.

That is, since luminance correction for a high-frequency componentregion of the image reduces the resolution of the image, it ispreferable not to apply luminance correction to the region, or to limitthe correction amount of luminance correction. Furthermore, sinceluminance correction for a low-frequency component region with a highluminance level of the image may give the user the impression thatdegradation in tone of the image has occurred, it is preferable not toapply luminance correction to the region, or limit a correction amount.Thus, Japanese Patent Laid-Open No. 2008-072604 proposes a method ofapplying luminance correction of a low-luminance region in a backlitscene by limiting it to a low-frequency component region.

To perform luminance correction for a low-frequency component regionwith a low luminance level as described in Japanese Patent Laid-Open No.2008-072604, it is necessary to extract the low-frequency componentregion by, for example, performing fast Fourier transform for an imagesignal obtained by capturing an image. When an image capturing apparatusshoots a still image, an image capturing circuit and correction circuitperform luminance correction for the still image to be recorded,according to the following procedure.

1. Develop the captured image signal

2. Convert the developed image into a luminance image

3. Extract a low-frequency component region from the luminance image

4. Decide a target luminance correction amount based on the luminancelevel of the low-frequency component region

5. Perform luminance correction for the low-frequency component regionof the developed image with the decided target luminance correctionamount

6. Record the image having undergone the luminance correction

If, however, successively captured and obtained images are sequentiallyinput to the correction circuit when, for example, a moving image isshot or live view is active, it is necessary to sequentially performluminance correction for the input images and output them. That is,timing constraints are imposed on processing from a shooting instructionto a recording operation unlike a still image, that is, a case in whichother images are input to the correction circuit.

In shooting a moving image, for example, it is necessary to execute, inparallel, in each frame, correction amount decision processing ofdeciding a target luminance correction amount based on a captured andobtained image, and recording processing of recording a frame image byperforming luminance correction with the target luminance correctionamount for a low-frequency component region of the captured and obtainedimage. In this case, the processes need to be individually performed indifferent lines at the same time. It is, however, not realistic toprovide, in each line, a circuit for extracting a low-frequencycomponent region from a luminance image, since this increases thecircuit scale, cost, or power consumption. Considering theabove-described problem when luminance correction is performed for ahigh-frequency component region, it is necessary to provide a circuitfor extracting a low-frequency component region on the recordingprocessing line side.

Although it is also possible to provide a circuit for extracting alow-frequency component region on the correction amount decisionprocessing line side, and holding information indicating a low-frequencycomponent region used for the correction amount decision processing andthen using it for the recording processing, in the following problemarises in that case. When the correction amount decision processing andthe recording processing are simultaneously executed, it is impossibleto perform correction processing after a target luminance correctionamount is decided unlike a still image because of the timingconstraints. The decided target luminance correction amount, therefore,can only be used for the next frame and subsequent frames. That is, ifan object is expected to move like a moving image, a low-frequencycomponent region may change between a frame used to decide thecorrection amount and that to undergo luminance correction whenperforming luminance correction using the target luminance correctionamount. Therefore, luminance correction may be performed for ahigh-frequency component region, thereby causing degradation in imagequality, as described above.

In an arrangement in which the circuit for extracting a low-frequencycomponent region is provided on the recording processing line side, whensuccessively captured and obtained, images subsequently undergoluminance correction, in which a target luminance correction amount isdecided based on the luminance levels of pixels which include those in ahigh-frequency component region. If luminance correction is performedusing the thus decided target luminance correction amount for alow-frequency component region of the developed image, the low-frequencycomponent region may not be appropriately corrected; for example, theregion may be overcorrected. Thus, a high-quality corrected image maynot be output.

SUMMARY OF THE INVENTION

The present invention has been made in consideration of the problems ofthe conventional technique. The present invention provides an imagecapturing apparatus which outputs an image as a preferred luminancecorrection result even when a target luminance correction amount isdecided without considering frequency components, a control method forthe image capturing apparatus, and a recording medium.

According to one aspect of the present invention, there is provided animage capturing apparatus comprising: an image capturing unit configuredto sequentially output images obtained by capturing an object; adecision unit configured to decide a luminance correction target amountbased on a luminance distribution of the image output by the imagecapturing unit; a specification unit configured to specify alow-frequency region of a target image output by the image capturingunit; a correction unit configured to perform luminance correction withthe target amount decided in advance by the decision unit for a pixel,among pixels included in the target image, which is included in thelow-frequency region specified by the specification unit and has aluminance value not larger than a predetermined value; an obtaining unitconfigured to obtain an effective correction amount based on a luminancedistribution of the target image before the correction and a luminancedistribution of the target image after the correction, which has beencorrected by the correction unit with the target amount decided inadvance; and an output unit configured to output the target image afterthe correction, wherein the decision unit decides a luminance correctiontarget amount for a next correction target image based on the luminancedistribution of the target image and a ratio of the effective correctionamount to the target amount decided in advance.

According to another aspect of the present invention, there is provideda control method for an image capturing apparatus, comprising: an imagecapturing step of sequentially outputting images obtained by capturingan object; a decision step of deciding a luminance correction targetamount based on a luminance distribution of the image output in theimage capturing step; a specification step of specifying a low-frequencyregion of a target image output in the image capturing step; acorrection step of performing luminance correction with the targetamount decided in advance in the decision step for a pixel, among pixelsincluded in the target image, which is included in the low-frequencyregion specified in the specification step and has a luminance value notlarger than a predetermined value; an obtaining step of obtaining aneffective correction amount based on a luminance distribution of thetarget image before the correction and a luminance distribution of thetarget image after the correction, which has been corrected in thecorrection step with the target amount decided in advance; and an outputstep of outputting the target image after the correction, wherein in thedecision unit, a luminance correction target amount for a nextcorrection target image is decided based on the luminance distributionof the target image and a ratio of the effective correction amount tothe target amount decided in advance.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments (with reference to theattached drawings).

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing the functional arrangement of adigital camera 100 according to an embodiment of the present invention;

FIG. 2 is a flowchart illustrating live view processing according to theembodiment of the present invention;

FIG. 3 is a flowchart illustrating luminance correction processingaccording to the embodiment of the present invention;

FIG. 4 is a timing chart showing each process of the live viewprocessing according to the embodiment of the present invention;

FIGS. 5A and 5B are graphs showing examples of luminance distributionsbefore and after luminance correction according to the embodiment of thepresent invention;

FIG. 6 is a table for explaining an effective correction amount obtainedby luminance correction according to the embodiment of the presentinvention;

FIG. 7 is a timing chart for explaining control of a luminancecorrection target amount based on a difference with respect to thecorrect exposure according to the embodiment of the present invention;

FIG. 8 is a timing chart for explaining control of a luminancecorrection target amount based on a face detection result according tothe embodiment of the present invention;

FIGS. 9A and 9B are views for explaining the conventional problemsassociated with luminance correction for each frequency component of animage;

FIGS. 10A, 10B, and 10C are views for exemplifying luminance correctionfor each frequency component of images; and

FIG. 11 is a graph showing the relationship between a target amount andan effective correction amount in luminance correction for eachfrequency component of the images.

DESCRIPTION OF THE EMBODIMENTS

Exemplary embodiments of the present invention will be described indetail below with reference to the accompanying drawings. Note that inan embodiment to be explained below, a case in which the presentinvention is applied to a digital camera as an example of an imagecapturing apparatus which can sequentially perform luminance correctionfor successively captured and obtained images will be described. Thepresent invention, however, is applicable to an arbitrary device whichcan perform luminance correction for sequentially output images.

<Functional Arrangement of Digital Camera 100>

FIG. 1 is a block diagram showing the functional arrangement of adigital camera 100 according to the embodiment of the present invention.

A control unit 101 is, for example, a CPU, which controls the operationof each block of the digital camera 100. More specifically, the controlunit 101 controls the operation of each block by reading out theoperation program of live view processing (to be described later) from aROM 102, mapping the program on a RAM 103, and executing the program.

The ROM 102 is, for example, a rewritable non-volatile memory, whichstores information such as parameters necessary for the operation ofeach block of the digital camera 100 as well as the operation program ofthe live view processing.

The RAM 103 is a volatile memory, which is used as not only a mappingarea for the operation program of the live view processing but also astorage area for temporarily storing intermediate data output in theoperation of each block of the digital camera 100.

Assume in this embodiment that processing in each block of the digitalcamera 100 is implemented by hardware. The present invention, however,is not limited to this, and the processing in each block may beimplemented by a program for executing processing similar to that ineach block.

An image capturing unit 105 includes, for example, an image sensor suchas a CCD or CMOS sensor, which photoelectrically converts an opticalimage formed by an imaging optical system 104 on the image sensor. Theimage capturing unit 105 applies A/D conversion processing to an analogimage signal obtained by photoelectrical conversion, and outputs thethus obtained digital image signal to an image processing unit 106.

The image processing unit 106 executes development processing includingcolor conversion processing and luminance correction processing for theinput digital image signal, and generates a live view screen to bedisplayed on a display unit 110 (to be described later), an image to berecorded in a recording medium 111, or a frame image of a moving image.The image processing unit 106 performs luminance correction for a pixelwhich has a luminance value falling within a predetermined low-luminancerange, and belongs to a low-frequency component region. In thisembodiment, the image processing unit 106 includes a correction regionextraction unit 108 (to be described later), and obtains information ofa region or pixel to undergo luminance correction from the correctionregion extraction unit 108.

The image processing unit 106 also generates an image which has notundergone luminance correction in addition to the image which hasundergone luminance correction in the developing processing. Acorrection target amount decision unit 107 (to be described later)decides information of a luminance correction target amount based on theimage which has undergone luminance correction and the image which hasnot undergone luminance correction. More specifically, the imageprocessing unit 106 generates a luminance image for each of the imagewhich has undergone luminance correction and the image which has notundergone luminance correction, in order to decide a luminancecorrection target amount. Then, the image processing unit 106 outputs,to the correction target amount decision unit 107 or a correctiondetermination unit 109, the luminance image of the image which hasundergone luminance correction, and that of the image which has notundergone luminance correction.

The correction target amount decision unit 107 decides a correctiontarget amount (EV value) for luminance correction in the imageprocessing unit 106. More specifically, for the luminance image of theimage which has not undergone luminance correction and has been input bythe image processing unit 106, the correction target amount decisionunit 107 obtains information of the number of pixels with a luminancefalling within the predetermined low-luminance range. The correctiontarget amount decision unit 107 decides a luminance correction targetamount based on the number of low-luminance pixels. The correctiontarget amount decision unit 107 outputs and stores information of thedecided luminance correction target amount in, for example, the RAM 103.Assume that information indicating the predetermined low-luminance rangemay represent an upper limit threshold such that a value below thethreshold is determined as a low luminance, and is stored in, forexample, the ROM 102.

Note that if the digital camera 100 has a face detection function ofdetecting a face region of a person, it is preferable to set a luminancecorrection target amount so as to obtain the correct exposure of theface region. On the other hand, if luminance correction is performed soas to obtain the correct exposure of a low-luminance region like abacklit scene where no person exists, the corrected image may lookunnatural to the user. In this embodiment, therefore, in view of thelatter situation, a luminance correction target amount is decided basedon the number of pixels of the low-luminance region. A method of settinga luminance correction target amount, however, is not limited to this,and the method may be changed as needed according to a shooting mode oruser settings.

The correction region extraction unit 108 extracts information of theposition of a region or pixel to undergo luminance correction in theimage processing unit 106. More specifically, the correction regionextraction unit 108 analyzes frequency components by, for example,performing fast Fourier transform for the digital image signal input tothe image processing unit 106, and extracts a low-frequency componentregion (low-frequency region). Note that by assuming that information ofa low-frequency band is stored in, for example, the ROM 102, thecorrection region extraction unit 108 obtains the information to extracta low-frequency region. Then, the correction region extraction unit 108refers to the luminance component of each pixel of the extractedlow-frequency region, and further extracts a pixel with a luminancefalling within the predetermined low-luminance range. The correctionregion extraction unit 108 outputs, to the image processing unit 106,the thus obtained information of the position of the low-luminance pixelincluded in the low-frequency component region.

Based on the luminance correction target amount and an actual correctionamount (effective correction amount) of the image which has undergoneluminance correction with the target amount, the correctiondetermination unit 109 evaluates the luminance correction with thetarget amount. The correction determination unit 109 analyzes theeffective correction amount obtained by luminance correction using theluminance image of the input image which has undergone the luminancecorrection and the luminance image of the input image which has notundergone the luminance correction. The correction determination unit109 determines based on the effective correction amount and thecorrection target amount used for the luminance correction whether theresult of the luminance correction with the target amount indicatesovercorrection, and then decides a coefficient (target amount adjustmentcoefficient) for adjusting the luminance correction target amount. Thecorrection determination unit 109 outputs and stores information of thedecided target amount adjustment coefficient in, for example, RAM 103.The thus decided target amount adjustment coefficient is also used todecide a luminance correction target amount by the correction targetamount decision unit 107.

The display unit 110 serves as the display device of the digital camera100, such as a small LCD. In the live view processing (to be describedlater) according to this embodiment, the image which has undergoneluminance correction by the image processing unit 106 is displayed onthe display unit 110. More specifically, after luminance correction bythe image processing unit 106 is completed, the image processing unit106 outputs the corrected image to the display unit 110 under control ofthe control unit 101. For the input corrected image, the display unit110 controls to read out data to be superimposed from a group of GUIdata stored in the ROM 102, and to superimpose it, thereby displayingthe obtained image.

The recording medium 111 is the internal memory of the digital camera100, or a recording device detachably connected to the digital camera100, such as a memory card or HDD. When the digital camera 100 shoots amoving image, the image which has undergone luminance correction in theimage processing unit 106 is recorded in the recording medium 111. Adescription thereof will be omitted in this embodiment. Assume in thiscase that the image processing unit 106 encodes the image havingundergone luminance correction in an encoding format according to therecording format of the moving image, and outputs the obtained image.

An operation input unit 112 is a user interface such as the releasebutton and menu button of the digital camera 100. Upon detecting theuser operation of the operation member of the digital camera 100, theoperation input unit 112 outputs a control signal corresponding to theoperation to the control unit 101.

<Live View Processing>

The live view processing of the digital camera 100 with such arrangementaccording to this embodiment will be described in detail with referenceto a flowchart shown in FIG. 2. It is possible to implement processingcorresponding to the flowchart when, for example, the control unit 101reads out a corresponding processing program stored in the ROM 102, mapsthe program on the RAM 103, and then executes the program. Assume thatthe live view processing starts when, for example, the digital camera100 is activated in a shooting mode.

In step S201, the control unit 101 executes AE processing, and decidesexposure conditions. More specifically, the control unit 101 causes aphotometry unit (not shown) to measure light from an object, and decidesexposure conditions such as a shutter speed, aperture value, andsensitivity setting based on an obtained measurement result.

In step S202, the control unit 101 causes the image capturing unit 105to capture the object. More specifically, according to the exposureconditions decided in step S201, the control unit 101 causes a timinggenerator (not shown) to generate a timing signal to instruct the imagecapturing unit 105 to perform exposure and to read out an analog imagesignal from the image sensor, thereby outputting the generated signal tothe image capturing unit 105. The image capturing unit 105 performsexposure, and reads out the analog image signal in response to thetiming signal. The image capturing unit 105 applies A/D conversionprocessing to the analog image signal, and outputs the thus obtaineddigital image signal to the image processing unit 106.

After step S202, the control unit 101 executes the following processesin steps S203, S204, and S205 in parallel.

In step S203, the control unit 101 executes AE processing similarly tostep S201, and decides exposure conditions for next exposure. In stepS204, the control unit 101 executes AF processing according to acontrast detection method, and controls drive of a focus lens includedin the imaging optical system 104. More specifically, the control unit101 detects the defocus amount of the object with respect to theobtained digital image signal, decides the driving position of the focuslens based on the defocus amount, and causes a focus lens driving unit(not shown) to drive the focus lens to the driving position.

In step S205, the control unit 101 executes luminance correctionprocessing to decide a luminance correction target amount, and todisplay, on the display unit 110, the image which has been applied withluminance correction. Upon completion of the processes in steps S203,S204, and S205 in one frame time, the control unit 101 returns theprocess to step S202 to expose a next frame.

(Luminance Correction Processing)

The luminance correction processing according to the embodiment will befurther described with reference to a flowchart shown in FIG. 3.

In step S301, the control unit 101 causes the correction regionextraction unit 108 to specify a pixel to undergo luminance correctionamong pixels contained in the digital image signal (target image) inputto the image processing unit 106. More specifically, the correctionregion extraction unit 108 specifies a low-frequency region by executingfast Fourier transform for the input luminance image. Furthermore, thecorrection region extraction unit 108 specifies a pixel, among thoseincluded in the specified low-frequency region, which has a luminancecomponent with a predetermined value or smaller, and outputs informationfor specifying the pixel to the image processing unit 106.

In step S302, the control unit 101 causes the image processing unit 106to execute development processing for the digital image signal. Thecontrol unit 101 first causes the image processing unit 106 to generatean image which has been applied with luminance correction and is to bedisplayed as a live view image on the display unit 110. Morespecifically, the control unit 101 obtains the information of theluminance correction target amount which has been decided in advance andstored in the RAM 103, and inputs the obtained information to the imageprocessing unit 106. The image processing unit 106 reads out, from theROM 102, a correction curve, corresponding to the luminance correctiontarget amount, for deciding a correction amount for the luminancecomponent of each pixel. For the correction target pixel which has beenspecified by the correction region extraction unit 108 in step S301, theimage processing unit 106 performs luminance correction with thecorrection amount which has been decided based on the correction curvecorresponding to the luminance correction target amount. In addition tothe image which has been applied with the luminance correction, thecontrol unit 101 causes the image processing unit 106 to generate animage which has not been applied with the luminance correction. That is,in this step, the image processing unit 106 executes the developmentprocessing of simultaneously generating an image which has been appliedwith the luminance correction and an image which has not been appliedwith the luminance correction. Note that it is assumed that the “last”decided luminance correction target amount is used as “the luminancecorrection target amount decided in advance”. The luminance correctiontarget amount, however, is not limited to this.

In step S303, the control unit 101 causes the image processing unit 106to transmit, to the display unit 110, the image which has been appliedwith the luminance correction and generated in step S302, and causes thedisplay unit 110 to display the image as a live view image.

Note that the luminance correction target amount used for the luminancecorrection in the development processing in step S302 is “the lastdecided luminance correction target amount”. In this embodiment, digitalimage signals which are successively captured and sequentially outputfrom the image capturing unit 105 are presented, with a minimum delay,as a live view image after performing luminance correction. A luminancecorrection target amount corresponding to the digital image signaloutput from the image capturing unit 105 is, therefore, decided after animage obtained by applying luminance correction to an imagecorresponding to the digital image signal is displayed, as shown in FIG.4. That is, a target amount used for the luminance correction in stepS302 is the latest target amount which has been decided by thecorrection target amount decision unit 107 and stored in the RAM 103 forthe digital image signal captured and obtained before the digital imagesignal output from the image capturing unit 105.

Referring to FIG. 4, after exposure, reading out of an analog imagesignal from the image sensor, and development processing are performedaccording to vertical synchronizing (VD) signals, a luminance histogramis generated, and a luminance correction target amount is decided. Inthis example, a luminance correction target amount decided for an imagecaptured in one frame is used for luminance correction for an imagecaptured two frames after that frame. The correction target image forwhich the decided luminance correction target amount is used is notlimited to this, and can be changed according to the processing speed ofthe control unit 101, the number of effective pixels of the imagesensor, or the number of pixels for which a luminance histogram isgenerated.

Note that in the example of FIG. 4, it is assumed that a value (EV=0)with which an image does not change before and after luminancecorrection is set as a target amount until a luminance correction targetamount is decided based on an actually captured and obtained digitalimage signal.

In step S304, the control unit 101 causes the image processing unit 106to generate a luminance image formed by luminance components byconverting pixel values for each of the image which has been appliedwith the luminance correction and the image which has not been appliedwith the luminance correction, both of which have been generated in stepS302. Under the control of the control unit 101, the image processingunit 106 outputs, to the correction target amount decision unit 107, theluminance image of the image which has not been applied with theluminance correction. Furthermore, under the control of the control unit101, the image processing unit 106 outputs, to the correctiondetermination unit 109, the luminance image of the image which has beenapplied with the luminance correction.

After step S304, the control unit 101 executes, in parallel, theprocessing in steps S305 and S306, and processing in steps S307 to S309.A unit which performs the processing in steps S305 and S306 is differentfrom that which performs the processing in steps S307 to S309. In thisembodiment, the correction target amount decision unit 107 executes theformer processing and the correction determination unit 109 executes thelatter processing.

In step S305, the control unit 101 causes the correction target amountdecision unit 107 to generate a luminance histogram for the image whichhas not been applied with the luminance correction. More specifically,the correction target amount decision unit 107 analyzes the luminancedistribution of the luminance image by referring to the luminance value(0 to 255) of each pixel of the luminance image, thereby generating aluminance histogram. Note that in this step, under the control of thecontrol unit 101, the correction target amount decision unit 107outputs, to the correction determination unit 109, the generatedluminance histogram of the image which has not been applied with theluminance correction.

In step S306, the control unit 101 causes the correction target amountdecision unit 107 to decide a luminance correction target amount. Morespecifically, the correction target amount decision unit 107 refers tothe luminance histogram generated in step S305, and decides a newluminance correction target amount based on the number of pixels with aluminance value equal to or smaller than a predetermined value which isdetermined as a low luminance.

On the other hand, in step S307, the control unit 101 causes thecorrection determination unit 109 to generate a luminance histogram forthe image which has been applied with the luminance correction. Uponreceiving the luminance histogram for the image which has not beenapplied with the luminance correction from the correction target amountdecision unit 107, the control unit 101 advances the process to stepS308.

In step S308, the control unit 101 causes the correction determinationunit 109 to compare the luminance histogram of the image which has beenapplied with the luminance correction with that of the image which hasnot been applied with the luminance correction, and to calculate aneffective correction amount. In this embodiment, the effectivecorrection amount is calculated by the following method.

(Calculation of Effective Correction Amount)

If, for example, the luminance histogram of the image which has not beenapplied with the luminance correction is as shown in FIG. 5A, thecorrection determination unit 109 obtains, for each predeterminedluminance value, the number of pixels with a luminance value which fallswithin the range from a minimum value 0 to the predetermined luminancevalue. In this embodiment, the predetermined luminance value isindicated by Y1 in FIG. 6. The correction determination unit 109 obtainsthe number (PixelNum) of pixels with a luminance value which fallswithin a range up to each luminance value.

The correction determination unit 109 refers to the luminance histogram(FIG. 5B) of the image which has been applied with the luminancecorrection to specify a luminance value (Y2) such that the accumulatednumber of pixels with a luminance value which falls within the rangefrom the minimum value 0 to the luminance value (Y2) reaches (exceeds)PixelNum. That is, the correction determination unit 109 analyzes anextended luminance range within which pixels distributed within aspecific luminance range before the luminance correction are distributedafter the luminance correction. Then, the correction determination unit109 calculates an effective correction amount ΔY obtained by theluminance correction for each predetermined luminance value accordingto:ΔY=log₂(Y2/Y1)

Calculating the effective correction amount by the APEX system makes iteasy to compare it with the EV value of the APEX system as a luminancecorrection target amount.

In step S309, the control unit 101 causes the correction determinationunit 109 to decide a target amount adjustment coefficient based on theeffective correction amount and the latest luminance correction targetamount used for the development processing in step S302. Morespecifically, the correction determination unit 109 determines whetherthe effective correction amount exceeds the latest luminance correctiontarget amount, that is, whether the effective correction amount withrespect to the latest luminance correction target amount exceeds 1. Thatis, if the effective correction amount exceeds the latest luminancecorrection target amount, overcorrection has occurred by the luminancecorrection. If the correction determination unit 109 determines that theeffective correction amount exceeds the latest luminance correctiontarget amount, it decides the target amount adjustment coefficient to be0, and stores it in the RAM 103; otherwise, it sets the target amountadjustment coefficient to 1, and stores it in the RAM 103.

In step S310, the control unit 101 causes the correction target amountdecision unit 107 to decide, by reflecting the target amount adjustmentcoefficient, a final target amount to be used for next luminancecorrection. More specifically, the control unit 101 reads out the targetamount adjustment coefficient from the RAM 103, and transmits it to thecorrection target amount decision unit 107. Then, the correction targetamount decision unit 107 decides a final target amount for nextluminance correction by multiplying the new luminance correction targetamount decided in step S306 by the input target amount adjustmentcoefficient, and completes the luminance correction processing. That is,if it is determined in step S309 that overcorrection has occurred, thetarget amount for next luminance correction is set to 0, and thusluminance correction is not performed.

With the above processing, it is possible to determine whether theresult of luminance correction indicates overcorrection, even if theluminance correction is performed using a target amount which has beendecided without limiting to a low-frequency region. If it is determinedthat the result indicates overcorrection, it is possible to control notto perform luminance correction, thereby avoiding a situation in whichimages with an image quality degraded due to overcorrection aresuccessively displayed or recorded.

Note that in this embodiment, if it is determined that the effectivecorrection amount exceeds the latest luminance correction target amount,correction is controlled not to be performed in the next luminancecorrection. The present invention, however, is not limited to this.Processing of deciding a target amount for the next luminance correctionmay be modified as follows, or may be controlled by combining thefollowing methods.

(1) Decrease a Target Amount in the Case of Overcorrection

If overcorrection occurs, the correction determination unit 109 maydecide the target amount adjustment coefficient to be 1 or smaller,thereby decreasing a target amount for the next luminance correction tobe smaller than the latest target amount.

(2) Do not Perform Correction when Correction Effects are not Expected

When the ratio of an effective correction amount to the latest luminancecorrection target amount is equal to or smaller than a predeterminedthreshold, correction effects are not expected even though luminancecorrection is performed for an image. In this case, the correctiondetermination unit 109 may decide the target amount adjustmentcoefficient to be 0, thereby controlling a next target amount to be avalue such that luminance correction is not performed. Alternatively,the correction determination unit 109 may decide the target amountadjustment coefficient to gradually approach 0, thereby controlling thenext target amount to be a value equal to or smaller than the latesttarget amount. Note that a case in which correction effects are notexpected even though luminance correction is performed includes, forexample, a case in which there are many high-frequency component regionsin an image, and a case in which the luminance of the whole image ishigh.

(3) Control a Target Amount Based on the Ratio of an EffectiveCorrection Amount to the Latest Luminance Correction Target Amount

If the ratio of an effective correction amount to the latest luminancecorrection target amount is not 1, this indicates that it is impossibleto obtain a desired correction result even though luminance correctionis performed using the target amount. As shown in FIG. 11, because theeffective correction amount is proportional to the luminance correctiontarget amount, it may be possible to obtain an almost preferredcorrection result by deciding a new target amount based on thedifference between the latest luminance correction target amount and theeffective correction amount. The correction determination unit 109 maycontrol the next target amount to be a value which causes the luminancecorrection result to converge to a preferred result by deciding thetarget amount adjustment coefficient to be the reciprocal of theeffective correction amount with respect to the latest luminancecorrection target amount.

(4) Control a Luminance Correction Target Amount Based on a Differencewith Respect to the Correct Exposure

As described above, AE processing is executed when live view display isperformed or a moving image is shot. If an amount By of exposure for amain object changes due to a change in the object or a light source, themain object may deviate from the correct exposure. If the amount ofexposure of the main object included in an image is not the correctexposure, the image may include a blown-out highlight region or ashadow-detail loss region. That is, even if a luminance correctiontarget amount is decided using a luminance image corresponding to animage which has not been adjusted to the correct exposure, a preferredcorrection result may not be obtained. If, therefore, the absolute valueof the difference between the correct exposure and the amount ofexposure of the main object within the image is equal to or larger thana threshold, the correction determination unit 109 may decide the targetamount adjustment coefficient to be 1 to hold the luminance correctiontarget amount without any change, as shown in FIG. 7. In the example ofFIG. 7, while the absolute value of the difference between the correctexposure and the amount of exposure of the main object is 1 or larger,the luminance correction target amount is held without any change.

(5) Control a Luminance Correction Target Amount Based on a FaceDetection Result

If the digital camera 100 has a function of detecting a face region, foran image where a face region has been detected, luminance correction isperformed to obtain a correction exposure of the face region. To thecontrary, for an image where no face region has been detected, because aregion for which the correct exposure is to be obtained cannot bespecified, luminance correction by analyzing a luminance distribution asdescribed above is performed. A result of detecting a face region,however, may change depending on the face direction of a person. Thatis, if, for an image captured after an image where a face region hasbeen detected, such as a next frame, luminance correction by analyzing aluminance distribution is performed when no face region is detected, apreferred correction result may not be obtained. Alternatively, since acorrection result for each frame is different, the user may have theimpression that the image flickers. If, therefore, no face region isdetected in an image captured after a face region is detected, thecorrection determination unit 109 may hold the luminance correctiontarget amount without any change, as shown in FIG. 8.

Note that depending on a change amount between the latest target amountand the target amount for next luminance correction, which has beendecided in consideration of the target amount adjustment coefficient, asudden change in luminance may occur in a luminance correction result,thereby causing the user to experience discomfort. If, therefore, thedifference between the target amount for next luminance correction andthe latest target amount exceeds a predetermined value defined as achange in luminance which does not cause the user to feel discomfort,the correction determination unit 109 may change the target amount fornext luminance correction so that the difference is equal to or smallerthan the predetermined value.

In the above description, for descriptive convenience, assume that theprocess returns to step S202 after each of the processes in steps S203,S204, and S205 of the above flowchart shown in FIG. 2, which areexecuted in parallel, is completed. Since, however, images for live vieware actually output from the image capturing unit 105 sequentially, theprocess may forcibly return to step S202 when, for example, one frametime has elapsed after start of the development processing, as shown inFIG. 4. In this case, the processing in step S304 and subsequent stepsof the luminance correction processing in step S205 is continuouslyexecuted even after one frame time has elapsed, as a matter of course.

As described above, the image capturing apparatus of this embodiment canoutput an image as a preferred luminance correction result even if atarget luminance correction amount is decided without consideringfrequency components.

Other Embodiments

Aspects of the present invention can also be realized by a computer of asystem or apparatus (or devices such as a CPU or MPU) that reads out andexecutes a program recorded on a memory device to perform the functionsof the above-described embodiment(s), and by a method, the steps ofwhich are performed by a computer of a system or apparatus by, forexample, reading out and executing a program recorded on a memory deviceto perform the functions of the above-described embodiment(s). For thispurpose, the program is provided to the computer for example via anetwork or from a recording medium of various types serving as thememory device (e.g., computer-readable medium).

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2011-199938, filed Sep. 13, 2011, which is hereby incorporated byreference herein in its entirety.

What is claimed is:
 1. An image capturing apparatus comprising: an imagecapturing unit configured to sequentially output images obtained bycapturing an object; a decision unit configured to decide a luminancecorrection target amount based on a luminance distribution of the imageoutput by said image capturing unit; a specification unit configured tospecify a low-frequency region of a target image output by said imagecapturing unit; a correction unit configured to perform luminancecorrection with the target amount decided in advance by said decisionunit for a pixel, among pixels included in the target image, which isincluded in the low-frequency region specified by said specificationunit and has a luminance value not larger than a predetermined value; anobtaining unit configured to obtain an effective correction amount basedon a luminance distribution of the target image before the correctionand a luminance distribution of the target image after the correction,which has been corrected by said correction unit with the target amountdecided in advance; and an output unit configured to output the targetimage after the correction, wherein said decision unit decides aluminance correction target amount for a next correction target imagebased on the luminance distribution of the target image and a ratio ofthe effective correction amount to the target amount decided in advance.2. The apparatus according to claim 1, wherein said decision unit sets,as a next target amount, a value obtained by multiplying the luminancecorrection target amount decided based on the luminance distribution ofthe target image by the reciprocal of the ratio of the effectivecorrection amount to the target amount decided in advance.
 3. Theapparatus according to claim 1, wherein when the ratio of the effectivecorrection amount to the target amount decided in advance is not largerthan a threshold, said decision unit decides the next target amount tobe a value not larger than the target amount decided in advance.
 4. Theapparatus according to claim 1, wherein said decision unit decides thenext target amount to be a value such that a difference between the nexttarget amount and the target amount decided in advance is not largerthan a predetermined value.
 5. The apparatus according to claim 1,wherein when the ratio of the effective correction amount to the targetamount decided in advance exceeds 1, said decision unit decides the nexttarget amount to be a value such that said correction unit does notperform luminance correction.
 6. The apparatus according to claim 1,wherein when the ratio of the effective correction amount to the targetamount decided in advance exceeds 1, said decision unit decides the nexttarget amount to be a value smaller than the target amount decided inadvance.
 7. The apparatus according to claim 1, wherein when an absolutevalue of a difference between predetermined correct exposure and anamount of exposure of a main object in the target image which has beencorrected with the correction amount decided in advance is not smallerthan a threshold, said decision unit decides the next target amount tobe a value such that said correction unit does not perform luminancecorrection.
 8. The apparatus according to claim 1, further comprising aface detection unit configured to detect a face region in an imageoutput by said image capturing unit, wherein when said face detectionunit does not detect a face region in an image output by said imagecapturing unit after an image in which said face detection unit hasdetected a face region, said decision unit sets a luminance correctiontarget amount to be equal to a target amount decided for the image inwhich the face region has been detected.
 9. A control method for animage capturing apparatus, comprising: an image capturing step ofsequentially outputting images obtained by capturing an object; adecision step of deciding a luminance correction target amount based ona luminance distribution of the image output in the image capturingstep; a specification step of specifying a low-frequency region of atarget image output in the image capturing step; a correction step ofperforming luminance correction with the target amount decided inadvance in the decision step for a pixel, among pixels included in thetarget image, which is included in the low-frequency region specified inthe specification step and has a luminance value not larger than apredetermined value; an obtaining step of obtaining an effectivecorrection amount based on a luminance distribution of the target imagebefore the correction and a luminance distribution of the target imageafter the correction, which has been corrected in the correction stepwith the target amount decided in advance; and an output step ofoutputting the target image after the correction, wherein in thedecision step, a luminance correction target amount for a nextcorrection target image is decided based on the luminance distributionof the target image and a ratio of the effective correction amount tothe target amount decided in advance.
 10. A non-transitorycomputer-readable recording medium recording a program for causing acomputer to execute each step of a control method comprising: an imagecapturing step of sequentially outputting images obtained by capturingan object; a decision step of deciding a luminance correction targetamount based on a luminance distribution of the image output in theimage capturing step; a specification step of specifying a low-frequencyregion of a target image output in the image capturing step; acorrection step of performing luminance correction with the targetamount decided in advance in the decision step for a pixel, among pixelsincluded in the target image, which is included in the low-frequencyregion specified in the specification step and has a luminance value notlarger than a predetermined value; an obtaining step of obtaining aneffective correction amount based on a luminance distribution of thetarget image before the correction and a luminance distribution of thetarget image after the correction, which has been corrected in thecorrection step with the target amount decided in advance; and an outputstep of outputting the target image after the correction, wherein in thedecision step, a luminance correction target amount for a nextcorrection target image is decided based on the luminance distributionof the target image and a ratio of the effective correction amount tothe target amount decided in advance.
 11. An image capturing apparatuscomprising: an image capturing unit configured to sequentially outputimage signals obtained by capturing an object; a decision unitconfigured to decide a luminance correction target amount based on aluminance distribution of the image signal output by said imagecapturing unit; a specification unit configured to specify alow-frequency region of a target image based on a target image signaloutput by said image capturing unit; a correction unit configured toperform luminance correction with the luminance correction target amountdecided in advance by said decision unit for a region in the targetimage, which is included in the low-frequency region specified by saidspecification unit and has a luminance value not larger than apredetermined value; and an obtaining unit configured to obtain aneffective correction amount based on a luminance distribution of thetarget image signal before the correction and a luminance distributionof the target image signal after the correction, which has beencorrected by said correction unit with the luminance correction targetamount decided in advance, wherein said decision unit decides a newluminance correction target amount based on the luminance distributionof the target image signal and the effective correction amount.
 12. Acontrol method for an image capturing apparatus, the control methodcomprising: an image capturing step of sequentially outputting imagesignals obtained by capturing an object; a decision step of deciding aluminance correction target amount based on a luminance distribution ofthe image signal output in the image capturing step; a specificationstep of specifying a low-frequency region of a target image based on atarget image signal output in the image capturing step; a correctionstep of performing luminance correction with the luminance correctiontarget amount decided in advance in the decision step for a region inthe target image, which is included in the low-frequency regionspecified in the specification step and has a luminance value not largerthan a predetermined value; and an obtaining step of obtaining aneffective correction amount based on a luminance distribution of thetarget image signal before the correction and a luminance distributionof the target image signal after the correction, which has beencorrected in the correction step with the luminance correction targetamount decided in advance, wherein in the decision step, a new luminancecorrection target amount is decided based on the luminance distributionof the target image signal and the effective correction amount.